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[57] ABSTRACT 

A multi-node server transmits world-wide- web pages to 
network-based browser clients. A load balancer receives all 
requests from clients because they use a virtual address for 
the entire site. The load balancer makes a connection with 
the client and waits for the URL from the client. The URL 
specifies the requested resource. The load balancer waits to 
perform load balancing until after the location of the 
requested resource is known. The connection and URL 
request are passed from the load balancer to a second node 
having the requested resource. The load balancer re-plays 
the initial connection packet sequence to the second node, 
but modifies the address to that for the second node. The 
network software is modified to generate the physical net- 
work address of the second node, but then changes the 
destination address back to the virtual address. The second 
node transmits the requested resource directly to the client, 
with the virtual address as its source. Since all requests are 
first received by the load balancer which determines the 
physical location of the requested resource, nodes may 
contain different resources. The entire contents of the web 
site is not mirrored onto all nodes. Network bottlenecks are 
avoided since the nodes transmit the large files back to the 
client directly, bypassing the load balancer. Client browsers 
can cache the virtual address, even though different nodes 
with different physical addresses service requests. 
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WORLD-WIDE-WEB SERVER WITH Internet-Protocol (IP) address. Each computer is typically 

DELAYED RESOURCE-BINDING FOR assigned a different IP address so that no two machines have 

RESOURCE-BASED LOAD BALANCING ON the same IP address. The IP address is often written as four 

A DISTRIBUTED RESOURCE MULTI-NODE decimal numbers separated by periods. Each decimal num- 

NETWORK 5 ber represents an 8-bit binary number, from zero to 255 in 

decimal notation. Thus a computer in IBM's domain might 

BACKGROUND OF THE INVENTION— FIELD have the IP address 209.180.55.2 while another computer in 

OF THE INVENTION that domain might have the address 209.180.55.103. 

~. . j ■ i ... i i Client Browsers Accessing Web Servers 

This invention relates to network servers, and more par- A . m e ° v . , i i- *u m 

' r io FIG. 1 is a diagram of a client browser looking up the IP 

ticularly to Internet Servers. r , , . c , . TrnT TT ? * 

J address of a host specified in a URL. Users of a remote 

BACKGROUND OF THE INVENTION computer use client software known as an Internet browser 

DESCRIPTION OF THE RELATED ART or smi P i y a browser. Popular browsers include Netscape 

Navigator by Netscape Communications, Inc. of Mountain 

Use of the global network known as the Internet has is View, Calif, and Internet Explorer by Microsoft Corp. of 

skyrocketed. Advertisers commonly feature their Internet Redmond, Wash., although many other browsers and other 

addresses in television, billboard, and magazine ads. Con- types of client software are used. 

sumers with a remote computer can access the Internet using Browser 10 initiates a communication session with a 

client software known as a browser. Explosive growth is remote server by the user selecting a URL, perhaps by 

occurring in the part of the Internet known as the World- 2 o mouse-clicking on a hyper link to a new web page. Host 

Wide Web, or simply the "web". The web is a collection of name 11, "www.round.com", in the URL "http:// 

millions of files or "web pages" of text, graphics, and other www.round.com/file.htmr', is sent to domain-name-system 

media which are connected by hyper-links to other web (DNS) server 14, which is a special Internet server with 

pages. These may physically reside on a computer system look-up table 16. DNS server 14 is often a special server at 

anywhere on the Internet — on a computer in the next room 25 an Internet Service Provider which contains most or all 

or on the other side of the world. domain names on the entire Internet, or in a local region of 

These hyper-links often appear in the browser as a graphi- the Internet. One DNS server may have to refer the request 

cal icon or as colored, underlined text. A hyper-link contains to another DNS server for unknown host-names, 

a link to another web page. Using a mouse to click on the DNS server 14 looks through look-up table 16 and finds 

hyper-link initiates a process which locates and retrieves the 30 an entry for the host www.round.com. This entry contains a 

linked web page, regardless of the physical location of that physical IP address 18 for the web-server host in the domain 

page. Hovering a mouse over a hyper-link or clicking on the round.com. This IP address 18 230.101.17.101 is returned to 

link often displays in a corner of the browser a locator for the browser 10. Browser 10 then stores this IP address in client 

linked web page. This locator is known as a Universal cache 20 for future use, a process known as browser caching 

Resource Locator, or URL. 35 of the IP address. 

Background of URL's, IP Addresses, HTML, HTTP Browser 10 then uses cached IP address 18' to initiate a 

The URL identifies a domain, a host within that domain, communication session with the remote computer which 

and sometimes a resource or file within a directory structure physically has the desired web page, the wwwround.com 

on the host computer. Domains can be thought of as a group server having the file.html file. FIG. 2 shows a browser using 

of computers, such as all computers on a company's net- 40 a cached IP address to retrieve a file from a remote server in 

work. For example, the domain "ibm.com" identifies a a server farm. Browser 10 reads the cached IP address 18' 

domain for the commercial company IBM, which may from client cache 20 and uses cached IP address 18' to 

include thousands of individual computers. Typically the initiate a communication session with remote server 22. 

URL identifies only those computers which are servers on Once the session with server 22 is established, URL 12 is 

the world-wide web by prefixing the domain with a host 45 sent to server 22. Server 22 then accesses disk 24 which 

name. Thus the URL "http://www.ibm.com" identifies an includes requested file 26, thefile.html web page. A file copy 

individual host computer within the ibm.com domain which 26* of requested file 26 is seat back to browser 10, which 

operates as a world-wide-web server for IBM. "HTTP" tells re-constructs the web page from file copy 26' and displays 

the host to use the hyper-text transfer protocol while deliv- the web page to the user. Other files such as graphic image 

ering files over the Internet. The files delivered can be from 50 files may also be transferred which were not directly 

resources such as database queries or execution of scripts by requested by the URL, but are referenced by the file,html 

the host as well as traditional files. file. 

A web server site may contain thousands of individual Server Farms for Large Web Sites Mirror Content 

web pages. The location of the file or resource containing a While some smaller web sites can be served from a single 

desired page is identified by appending a directory-path file 55 computer, larger web sites require multiple computer 

name to the host and domain names in the basic URL to form machines acting as servers. Some web sites receive as many 

a new URL. Thus the URL "http://www.ibm,com/dira/dirb/ as one million requests or "hits" per hour, requiring many 

dirc/intro.html" identifies a hyper- text markup-language workstation computers. 

(HTML) file called "intro.html" which resides on a host FIG. 2 shows server farm 30 which contains server 22 

named "www" within the ibm.com domain. The file resides 60 serving browser 10, and servers 22A, 22B, 22C which are 

in the dira directory and the dirb/dirc subdirectory. Often this servicing other browsers (not shown). Servers 22A, 22B, 

HTML file contains references to other files which are 22C each contain their own disks 24', each with a copy of all 

loaded automatically by the client's browser. the web pages in the site, including requested file 26. Server 

While the URL is used to locate a file on a host within a farm 30 is basically a group of replicated servers which can 

domain, it does not contain a physical address for the host 65 service requests from multiple browsers. Each server has a 

computer. Addresses of computer machines on the Internet copy of the entire web site. Any server can service any 

are specified using a 32-bit numeric identifier known as the request since the content is "mirrored" on all servers. 
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Each machine typically has its own unique TP address. Browsers attempting to use this IP address and connect with 

Since a domain can have many computer machines with the crashed server receive no response from the wwwround- 

many IP addresses, some way to provide to a client one of .com web site. These browsers are frozen out of the 

the many server machines' IP address is needed. One simple www.round.com web site. 

approach is known as rotating DNS or DNS round-robin 5 since the browser itself caches the IP address from the 

load-balancing. Dr^s server until the browser application is closed, browsers 

DNS server 14 of FIG. 1 contains look-up table 16 which can still attempt l0 access a cras hed server after the crash has 

is used to return IP addresses to host-lookup request from 0CCUrrccL na 3 shows a browscr using a cached IP addrcss 

client browsers, l^ok-up table 16 contams entries for dif- to access a crashed servef which fc fl0t ^ Browser 

ferent host names The entry for a host name specifies the P JQ 10A had previously cached IP address 18c for server 22C 

addresses for that host and each entry can contain several IP for ^ ^ roimdcom host . when browseT 10A attempts to 

addresses for that host. The entry for www.round.com host connect lQ vwwj OUndtCom server 2 2C is accessed. No 

on the domain round.com contains four IP addresses: response is received from server 22C since the server is not 

230.101.17.100 functioning. To Browser 10A, the web site www.round.com 

230.101.17.101 !S appears to be non-functional, even though to another 

230.101.17.102 browser 10, the web site is functional. 

230.101.17.103 Though the user of browser lOAmay repeatedly try to 
for the four servers 22A, 22, 22B, 22C of server farm 30 connect to the www.round.com web site, each time no 
serving the www.round.com web site. When a client response is received until server 22C is fixed. Since DNS 
requests a DNS look-up, one of these IP addresses is chosen 2D server 14 of FIG. 1 may continue to use the IP address of the 
in a round -robin fashion. Each time a different client looks crashed server 22 C, many users may be locked out from the 
up the host www.round.com, a different IP address is web site, even though other users can access the site, 
returned until all the available IP addresses are used. Then When browser 10A also caches IP address 18C, the 
the first IP address is returned again. Thus the first browser browser may not be informed that the IP address is no longer 
is sent the IP address for server 22A, the second browser is 25 valid even after DNS server updates its own cache. These 
sent the IP address for server 22, the third browser sent the browser caches may persist for several hours, preventing the 
IP address for server 22B, and the four browser sent the IP user from accessing the web site. Should the server 22C be 
address for server 22C. The fifth browser request to DNS removed from service permanently, perhaps being 
server 14 is sent the first server 22A, and so on in a re-assigned to another web site, the user is effectively 
round-robin fashion. 30 blocked from accessing the web site until the user flushes his 

Each DNS server operates independently of other DNS IP cache, which may not occur until the user exits the 

servers. Thus optimal load balancing is not always achieved. browser application. 

Other more sophisticated assignment schemes have been Of course, with a large server farm, the loss of one server 

used, such as "load-balancing DNS" which sends requests to blocks out only 1/N of the users, where N is the number of 

servers based on a balancing algorithm which attempts to 35 servers in the server farm. Thus for FIG. 3, one-fourth of the 

balance the load on each server. With this approach more current users are blocked out while V4hs of the current users 

powerful servers could be assigned more requests than have access to the web site. One-fourth of the new users 

weaker servers. looking up the host on a DNS server which still uses the old 

IP Addresses of Servers Cached on DNS Server IP address of the crashed server are also blocked from the 

DNS servers 14 (FIG. 1) often cache the results of 40 web site, 

domain-name lookups which were passed or forwarded to Router-Based Web Site 

other DNS servers for completion. The administrator of the An approach which mitigates some of these problems 

www.round.com web site has no way of actively updating inserts a multiplexer or router between the browser clients 

the contents of many DNS caches containing IP addresses of and the server farm. FIG. 4 illustrates a router-based server 

servers in server farm 30. Instead, the administrator must 45 farm. A single IP address of router 32, 230.101.17.200, is 

rely on the remote DNS servers periodically flushing their available to all DNS servers as the single IP address for the 

own cached IP addresses and looking up the www.round- web site. Browser 10 caches this IP address as cached IP 

.com host again. DNS servers may flush their cached IP address 34. Requests from browser 10 are sent to router 32 

addresses every few minutes or not for several weeks. IP since cached IP address 34 points to router 32. 

addresses can thus remain in a DNS server's cache long after 50 Router 32 receives all packets in the transmission from 

the server with the cached IP address is removed from browser 10. Router 32 might be a dedicated personal com- 

service. The IP address of the removed server can continue puter (PC) which uses an algorithm to determine which of 

to be assigned by the DNS server until the cached entry is servers 36A, 36, 36B, 36C in server farm 38 should service 

replaced or flushed. the request from browser 10. Router 32 may use a fairly 

For the example in FIG. 3, when server 22C crashes, its 55 complex load-balancing scheme which takes into account 

IP address 230.101.17.103 remains in use in DNS server requests from other browsers and the capability of each 

caches. Users that look-up the www.round.com host name server when some servers are powerful workstations while 

can be assigned the IP address of crashed server 22C. Users other servers are older, slower PC's, 

sent the IP address of crashed server 22C are unable to All the packets in the session from browser 10 received by 

access server farm 30, even though several other servers 60 router 32 are re-transmitted to server 3 6, with the destination 

22A, 22, 22B at server farm 30 are operational. IP address changed to the IP address for server 36, 

DNS Caching Blocks Some Users From Partially-Crashed 230.101.17.101. Server 36 retrieves the requested file 26 

Web Site from its local disk 24 and transmits it back to router 32, 

Several hours or even days may be required to flush the which then re-transmits the file to browser 10. 

IP address of the crashed server 22C from all DNS caches. 65 WheD a server crashes, such as crashed server 36C, only 

Thus DNS servers can continue to send the IP address of the those browsers which are currently connected to server 36C 

crashed server to browsers long after the server has crashed. experience server failure. Client caching of the router's IP 
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address causes all new sessions to be routed to router 32; that fewer users experience a browser lock-up when a server 

only sessions in progress to crashed server 36C receive no at the web site fails. A web site that can use the standard 

response from the web site. Thus when one of the servers DNS mechanism is desired to overcome the limitations of 

fails, only 1/N of the currently active requests fail, where N DNS caching and complex maintenance of round-robin 

is the number of servers. New requests do not fail since 5 DNS. 
router 32 detects when crashed server 36C is not functioning 
and no longer directs new requests to the down server. 

A commercial embodiment of a router-based web server A web site sends resources to a browser on a client 
has been announced by SOS Corp. of New York, N.Y., under connected to a computer network. The web site has a 
the name "Hydra WEB", and product literature indicates that 10 network connection point for receiving incoming data pack- 
a patent is pending. A second commercial embodiment is the ets from the computer network and for transmitting outgoing 
Cisco Local Director, manufactured by Cisco Systems of data packets to the computer network. A local network is 
San Jose, Calif. Each server 36A, 36, 36B, 36C contains a coupled to the network connection point and transfers data 
local copy of all content on the web site on disks 24, 24\ packets. A plurality of network nodes contain web servers 
Mirroring the full content of the site to all servers is a is with resources. The plurality of network nodes is connected 
disadvantage for web sites with a large amount of content, to the local network, The plurality of network nodes transmit 
because of the size and cost of the local disks. Certain web the resources as outgoing data packets over the local net- 
applications such as multimedia and video delivery can work to the network connection point through the computer 
require a particularly large amount of disk space. These network to the client. 

applications are expensive to implement and thus minimiz- 20 ^ balancer network node contains a load balancer that 

ing the number of copies at the server farm is desirable. receives the incoming data packets transmitted over the local 

Another disadvantage with the router web site is that all network from the network connection point. The load bal- 

data transfers go through router 32. Since many web pages determines an assigned server in the plurality of 

contain graphics or even video or sound, the amount of data network nodes to respond to a request from the client 

transferred from the server through the router to the browser 25 contained in an incoming data packet. The load balancer 

is large. Router 32 must be fast and efficient to handle load transfers a connection to the client to the assigned server, 

balancing and routing of incoming and outgoing packets. As m balancef network node me load 5alancer 

the web site becomes more popular and traffic grows, router b connected to ^ network connection point by the local 

32 can quickly become a bottleneck and limit performance netwQrk which {& alsQ connected t0 the luralit of network 

of the web site. Router 32 is also a single point of failure. 30 nodes ^ the incoming data packets are routed t0 the 

Load-Balancing Granularity Determines Users Affected by bt ^ ausm network node but outgoing data packets bypass the 

Server Failure rL . balancer network node. 

For round-robin DNS, the IP address of the web server is .» *. « 4 tl _ , . , , . . , u 

, . V . . , t1 . . In further aspects the balancer network node is in the 

assigned once to the client browser and all subsequent , ^ <? * i j 4 • ■ . ^ , 

*u- m j j *u u > i' * u plurality of network nodes containing web servers. The web 

accesses use this IP address until the browser s client cache 35 *V . J ,< L1 , r jj c h u 

, ™ . . r ♦ t i i a u t • 1 * Slte is addressable by one network address for all web 

is purged. This is client-level load-balancing granularity, . it _ , - , , , . . . . 

. 6 . . . A u- f ii servers m the plurality of network nodes containing web 

since each client is assigned one server machine for all r , \_ , J , . t 1% - . , j 

* c .t_ * i • * wt_ *t_ . j . servers. Each network node in the plurality of network nodes 

requests from that client. When the assigned server crashes, . , . * „ , *u u •* 

v . ... , , i , c ii a * contains only a portion of all the resources at the web site; 

the clients using that server are blocked for all future 1( 4 £ • . , 4 . . . „ . / 

♦•i a. r * v ** -i a . n all resources at the web site are not mirrored to all network 

requests until the client application is closed. 40 , t ... 

™ . i j f . 4 , 4 i ii j nodes at the web site. 

The router-based web site has request-level load- 
balancing granularity. Servers are assigned to handle indi- In stm ^ nhGT of ^ invention a content means 
vidual requests from browsers. When the assigned server stores an "^cation of which network nodes in the plurality 
crashes, the outstanding requests to the server are blacked of network nodes contain each «»ouk». A URL means 
out but clients can still access other servers in the farm. 45 receives incoming data packets from the client which con- 
Server Problems Plague the Internet tain a rc£ l ucst for a ^source. A requested resource is deter- 

Many Internet users can testify to the utter frustration from tne ^coming data packets. Compare means is 

when the "SERVER NOT RESPONDING" error message is C0U P led to the content means ™ d 15 coupled to the URL 

displayed on their browser while trying to connect to a web means - li compares the requested resource to the indication 

site. Users often blame the company which administers the 50 of whic h network nodes in the plurality of network nodes 

unavailable web site. Web sites are not as fault-tolerant as contain each resource. A list of network nodes containing the 

possible despite large investments in replicated servers. An requested resource is outputted. 

intelligently-designed web-site architecture with better Balancing means receives the list of network nodes con- 
fault-tolerance is needed. taining the requested resource. It chooses as an assigned 
It is desired to reduce the frequency of "SERVER NOT 55 node one of the network nodes in the list of network nodes. 
RESPONDING" messages that Internet users often receive. Tnns tne load balancer chooses an assigned node based on 
While many web sites use server architectures such as DNS the resources contained by each network node. The load 
round-robin and router-based load-balancing, a more effi- balancer performs resource-based load balancing, 
cient and fault-tolerant web-site architecture is desired. It is In other aspects a delay means in the load balancer delays 
desired to avoid the data bottleneck and single point of 60 assignment of the assigned node until an incoming data 
failure at the router for router-based web sites. It is also packet containing the request for the resource is received, 
desired to use inherent characteristics of web traffic to more Thus load balancing is delayed. 

efficiently design a web-site architecture. Mirroring the In other aspects the invention is a method which makes a 

content of the entire web site to all servers at the site is connection and sets up a session between the client and a 

undesirable, but having differing content on different servers 65 load balancer at a web site. The load balancer waits for a 

is desired while still performing load balancing. A web site URL request from the client once the load balancer has made 

with request-level load -balancing granularity is desired so the connection with the client. It receives the URL request 
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from the client and decodes the URL request to determine a 
requested resource. An identifier for the requested resource 
is compared to identifiers for resources located on a plurality 
of nodes and a first subset of the plurality of nodes is 
determined which contain the requested resource. The URL 
request is assigned to an assigned node in the first subset of 
the nodes which contain the requested resource and the 
connection and the session setup are transferred to the 
assigned node which contains the requested resource. The 
assigned node reads the requested resource and transmits the 
requested resource to the client. Thus the assigned node is 
selected based on a location of the requested resource 
determined from the URL request. 

In further aspects the packets received from the client are 
TCP/IP packets having a destination IP address which is a 
virtual IP address of the load balancer. The virtual IP address 
of the load balancer is changed in the packets to a real IP 
address of the assigned node and the packets are passed to 
a modified ' IP layer. The real IP address determines a 
physical route from the load balancer to the assigned node 
over a network and a physical network address is generated 
for the assigned node and the physical network address is 
attached to the packets. The real IP address in the packets is 
changed back to the virtual IP address of the load balancer 
before transmission of the packets with the physical network 
address. Thus the physical network address is generated 
from the real IP address of the assigned node, but the packets 
are transmitted to the assigned node containing the virtual IP 
address of the load balancer. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagram of a client browser looking up the IP 
address of a host specified in a URL. 

FIG. 2 shows a browser using a cached IP address to 
retrieve a file from a remote server in a server farm. 

FIG. 3 shows a browser using a cached IP address to 
access a crashed server which is not responding. 

FIG. 4 illustrates a router-based server farm. 

FIG. 5 highlights the asymmetric nature of data transfer at 
a world- wide-web site. 

FIG. 6 is a diagram of a web server which asymmetrically 
routes incoming traffic through a load-balancer while 
bypassing the load-balancer for files transmitted back to 
client browsers. 

FIG. 7 is a diagram of a TCP/IP packet transmitted 
through a local-area network (LAN). 

FIG. 8 is a diagram illustrating TCP state migration of a 
connection from the load balancer to a server node, 

FIG. 9 is a flowchart of processing a browser request by 
a prior-art router-based load balancer such as shown in FIG. 
4. 

FIG. 10 is a flowchart showing load balancing and state 
migration delayed until the connection is made and the URL 
request received. 

FIG. llAis a chart illustrating packets transferred among 
the browser, load balancer, and the assigned server when 
establishing the connection and transferring the connection 
to the assigned server which responds to the URL request. 

FIG. 11B shows the browser sending a second URL 
request, PUSH(l), to the load balancer, which is passed 
through to the assigned server as PUSH(l)'. 

FIG. 12 is a diagram of network layers showing a packet 
sent from the client to the server which is intercepted by the 
load balancer. 
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FIG. 13 is a diagram of network layers transmitting 
packets from the server to the client browser, 

FIG. 14 is a diagram of modifications to the IP layer for 
the load balancer's node. 

FIG. 15 is a flowchart for a modified IP layer input 
module. 

FIG. 16 is a flowchart of an IP layer output module which 
is modified for transmitting packets from the load balancer. 
} FIG. 17 highlights that the real IP address of the assigned 
server is used when multiple hops are required. 

FIG. 18 is a flowchart of the operation of the load 
balancer. 

FIG. 19 is a diagram of a fault-tolerant web site with a 
is back-up load balancer and dual Internet connections. 

DETAILED DESCRIPTION 

The present invention relates to an improvement in Inter- 
net and Intranet server farms. The following description is 

20 presented to enable one of ordinary skill in the art to make 
and use the invention as provided in the context of a 
particular application and its requirements. Various modifi- 
cations to the preferred embodiment will be apparent to 
those with skill in the art, and the general principles defined 

25 herein may be applied to other embodiments. Therefore, the 
present invention is not intended to be limited to the par- 
ticular embodiments shown and described, but is to be 
accorded the widest scope consistent with the principles and 
novel features herein disclosed. 

30 The inventors have realized that most traffic at web sites 
is inherently asymmetric. Users download huge amounts of 
data and graphics from a web site, but provide only small 
amounts of data in the form of requests to the web site. The 
amount of data flowing out of a web site far surpasses the 

35 trickle of data flowing into the site. 

FIG. 5 highlights the asymmetric nature of data transfer at 
a world-wide-web site. Client browser 10 operates an Inter- 
net browser application which connects to server 22 in 
server farm 30. Client browser lOAalso operates an Internet 

40 browser application which connects to server 22C in server 
farm 30, and other browsers (not shown) may also be 
connected to this and other servers 22A, 22B. 
Browser 10 imbeds requests and commands and a small 

45 amount of data in URL's, which are transmitted to server 22. 
Each URL contains about 50 to 150 bytes of information, 
excluding IP addresses and packet headers and other net- 
work overhead. A URL often contains information other 
than a requested file. For example, when the user of browser 

50 10 mouse-clicks on a bitmap image displayed on a web 
page, the relative coordinates of the mouse's location when 
the mouse click occurred are included in the URL: http:// 
www.round.com/cgi-bin/coo. cgi?102,315. Server 22 
decodes the coordinates in the URL and determines where 

55 on the web page the user mouse -clicked. 

The server then performs the action requested, such as 
opening and transmitting another web page to browser 10. 

Other information sometimes embedded in the URL 
includes passwords or search text that the user types in, or 

eo name and address information typed in. Since the amount of 
data that a user types in is limited, the number of bytes for 
this data is small. 

Traffic from server 22 to browser 10 consists of large data 
files which are used to reconstruct the web pages on browser 

65 10. Since many web sites feature colorful graphics, large 
graphics files are typically transmitted from server 22 to 
browser 10. As new types of media become commonplace, 
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sound, animation, and video files will accompany the graph- 
ics and HTML files being transmitted from server 22 to 
browser 10. Smaller web pages may result in only 10 
K-bytes of data returned to browser 10, while more graphi- 
cal web pages cause larger files to be transmitted, perhaps to 
several mega-bytes. Video and sound file can easily exceed 
several mega-bytes. 

Ihe result is that only 50 to 150 bytes of data are typically 
transmitted from browser 10 to server 22, while tens or 
hundreds of thousands of bytes of data are transmitted in the 
reverse direction, from server 22 to browser 10. In the near 
future, when sound and video become common, the reverse 
traffic will increase to several mega-bytes per 100-byte 
request. At that point the bandwidth of the reverse traffic will 
be about a thousand times the bandwidth from browser to 
server. 

Outgoing Data Bypasses Load-Balancer 

FIG. 6 is a diagram of a web server which asymmetrically 
routes incoming traffic through a load -balancer while 
bypassing the load-balancer for data transmitted back to 
client browsers. Browsers 10, 10A cache a virtual IP address 
34 in client cache 20. Virtual address 34 (230.101.17.200) is 
an IP address that identifies all servers at the web site. Unlike 
a conventional IP address which is unique to an individual 
host or server, the virtual IP address identifies the web site 
in its entirety. 

External routers are configured to deliver all incoming 
URL's and network traffic from browsers 10, 10A to load- 
balancer 54, which has the virtual IP address. 

Load-balancer 54 keeps track of which requests are being 
processed by each server in server farm 50, and attempts to 
balance the load of requests among the servers. As is 
subsequently discussed in more detail, load-balancer 54 
establishes the connection with browser 10 and waits for the 
URL before performing load balancing and assigning the 
request to a server. The connection and the URL request is 
then migrated to the assigned server. For example, the 
request from browser 10 is assigned and migrated to server 
52, while the request form browser 10A is assigned and 
migrated to server 52C. 

Unlike a router-based web site, the IP addresses of packets 
are not changed to the assigned server's local IP address. 
Instead, each server 52A, 52, 52B, 52C is assigned an 
additional IP address, the virtual IP address. A low-level 
Network-Interface Card (NIC) address is used to route the 
packets to the assigned server. Each server accepts any 
packet with either its local IP address or the virtual IP 
address. Incoming packets from the Internet backbone are 
routed to load-balancer 54 because they are given the NIC 
address of load balancer 54 by the Internet connection router 
(not shown). 

Server 52 sends HTML files for the web page back to 
browser 10 by using the source IP address in the packets 
received from load-balancer 54. Since load-balancer 54 does 
not modify the higher-level TCP/IP packets transmitted to 
server 52, server 52 appears to be receiving these packets 
directly from browser 10. Server 52 uses the browser's IP 
address as the destination IP address for outgoing packets, 
but the virtual IP address as the source address of these 
packets. Outgoing packets do not go through load-balancer 
54. The bandwidth of traffic through load balancer 54 is 
much less than through router 32 of FIG. 4 since only the 
relatively small incoming requests are routed through load 
balancer 54 while outgoing data bypasses load balancer 54. 

Load balancer 54 can handle multiple requests from 
multiple browsers since each request is relatively small. 
Outgoing packets from each server do not have to have their 
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IP addresses altered since each server uses the virtual IP 
address as the source address of outgoing packets. Thus the 
larger outgoing traffic of large mega-byte-sized files is 
bypassed around load-balancer 54. Multiple connections to 

5 the Internet can be provided for the outgoing traffic, since a 
single router/load-balancer is not required for all traffic. 
NIC Address Encapsulated in TCP/IP Packet 

FIG. 7 is a diagram of a standard TCP/IP packet trans- 
mitted through a local-area network (LAN). Packet 180 

10 contains NIC address 182, which contains the low-level NIC 
address (e.g. Ethernet's media-access-controller (MAC) 
address) of the next destination station in the route to the 
final destination indicated by destination IP address field 
186. Several destination stations may exist on the route to 

is the station having the destination IP address. Each interme- 
diate station reads the destination IP address and determines 
the next station in the route. The NIC address of this next 
station is changed to the NIC address for the next leg of the 
route, but the IP addresses are not changed. 

20 The source station's IP address is contained in source IP 
address field 184, while the packet's type or protocol is 
included in protocol field 188. For Internet packets, the 
protocol is TCP, but other protocols may be used for local 
LAN traffic. The data being transmitted by the packet is 

25 contained in data field 189. A frame checksum (not shown) 
may be appended. Source IP address field 184, destination IP 
address field 186, and protocol field 188 are the IP header 
attached to the data by the IP layer, while NIC address 182 
is attached by the data-link layer. Packet 180 includes a 

30 TCP/IP header which includes flags indicating the type of 
packet (SYN, ACK, PUSH, RST, FIN). A sequence number 
is also included in the TCP/IP header to keep track of 
packets received. 

Larger data files must be broken into several packets 

35 which are transmitted separately and then re- assembled. A 
typical Ethernet packet can contain up to 1500 bytes. 
TCP Connection Migration — FIG. 8 

FIG. 8 is a diagram illustrating TCP state migration of a 
connection from the load balancer to a server node. Browser 

40 10 connects through Internet 66 to load balancer 70 and 
sends a URL request 102 once the connection 100 is made. 
Load balancer 70 does not have to be a separate, dedicated 
router or PC, and is shown as a software application running 
on server 56. Load balancer 70 can use many variations of 

45 balancing algorithms to determine which server 56, 51, 52 
should service the new URL request 102. Load balancer 70 
determines that the request should be assigned to server 52. 
The connection and URL request are migrated from load 
balancer 70 to server 52 using TCP state migration 120. 

50 Server 52 accesses disk 62 to read requested file 26 and 
sends a copy of requested file 26 to browser 10 through 
Internet 66 as data transfer 104. 
Resource-Based Load Balancing 

Each server 56, 51, 52 has a local disk 58, 60, 62, which 

55 contains different content. For example, only disk 62 con- 
tains requested file 26. Load balancer 70 maintains a direc- 
tory table of the locations of different files and resources on 
the web site. Load balancer 70 determines that only server 
52 and not servers 56, 51 can handle the request. The entire 

60 content of the web site does not have to be mirrored to each 
server's hard disk as in the prior art. Some of the more 
frequently accessed files and resources, such as the home 
page, may be replicated to all servers, but less-frequently 
accessed files and pages may be located on a single node or 

65 a few nodes. Other web sites resources may include dedi- 
cated servers with specific resources such as databases of 
built-in application-programming interfaces (API's) to inter- 
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face with user-defined programs, or software licenses to run 

particular programs on particular servers. 

Prior Art Load Balancing Performed Before URL is Parsed 

FIG. 9 is a flowchart of processing a browser request by 
a prior-art router-based load balancer such as shown in FIG. 
4. The browser first attempts to make a connection with a 
server at the web site by sending a SYN packet which 
requests that a connection be made, step 150, A SYN packet 
has its SYN flag set in the TCP header. The router/load 
balancer receives this request and performs load balancing 
based on the load of each server in the server farm. Load 
balancing selects an assigned server for this new request, 
step 152. The router/load balancer then forwards packets to 
the assigned server, step 154. 

The assigned server then makes the connection with the 
browser, step 156, by returning an SYN/ACK acknowledge 
packet to the browser by first routing it through the router. 
The browser responds with an ACK packet and then with a 
URL request, step 158. This URL packet is received at the 
router and then re- transmitted to the assigned server. The 
assigned server parses the URL request to determine which 
files are requested and then transmits these files back to the 
browser through the router/load balancer, step 160. 

Steps 150, 152, 154 are performed by the router/load 
balancer while steps 156, 158, 160 are performed by the 
assigned server. The router passes packets from the server 
through to the client's browser. 

URL File Request Arrives After Load Balancing in Prior Art 
Since load balancing is performed as soon as the first SYN 
packet is received, before the URL is sent, such prior-art 
load balancing cannot take into account the resource or file 
which is requested by the URL. All servers must have the 
same content, since the URL arrives after the server assign- 
ment has been made. The invention overcomes these limi- 
tations by delaying the assignment of the server until after 
the URL has been received. 

Simple router-based load balancing does not allow for 
re source -based load balancing because the requested 
resource or file is not known when the connection is first 
made. The requested file is known once the URL is received 
and parsed, which occurs after the connection is made. The 
browser does not send the URL until the connection has 
been made and the server responds with an acknowledgment 
packet. The router approach of FIGS. 4, 9 forwards all 
packets to the assigned server, and the assigned server then 
makes the connection and sends the acknowledgment. Once 
the connection is made with the assigned server, then the 
browser sends the URL, which is forwarded by the router to 
the assigned server. 

Delayed Load Balancing and TCP State Migration 

FIG. 10 is a flowchart showing load balancing and state 
migration delayed until after the connection is made and the 
URL request received. The TCP connection 100 is made 
between the browser and the load balancer by exchanging 
SYN and ACK packets. Once this connection is made, the 
load balancer waits until the browser sends the URL which 
indicates which file or resource is requested. Once the URL 
request 102 is received, the load balancer parses the URL to 
determine which resource is being requested. Based on the 
resource requested from parsing the URL, the load balancer 
determines which servers are best suited to serve the request. 
The load balancer then performs load balancing among the 
servers that can serve the request, step 125. 

The load balancer then transfers the connection and the 
current TCP state to the assigned server, using TCP state 
migration 120. TCP state migration is not simply forwarding 
packets through as they are received. Instead the packets 
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received are stored by the load balancer and then played 
back to the assigned server. The assigned server accesses its 
local disk to read the requested file and sends a copy of the 
requested file to the browser through the Internet as data 

5 transfer 104. 

TCP State Migration— FIG. 11A 

FIG. UAis a chart illustrating packets transferred among 
the browser, load balancer, and the assigned server when 
establishing the connection and transferring the connection 

10 to the assigned server which responds to the URL request, A 
browser application running on a remote client initiates a 
connection by sending a synchronizing packet, SYN(0), to 
the virtual IP address of the web site, Hie SYN(0) packet is 
routed to the load balancer since it is an incoming packet, 

is and the load balancer sets up a session with the browser by 
setting aside memory space and creating an entry in a 
session table. The load balancer replies with a SYN/ACK 
packet to the browser, and the browser replies with an 
acknowledgment packet, ACK(0). The SYN packet contains 

20 an initial sequence number which is determined by the 
browser's OS. The SYN/ACK packet contains acknowledg- 
ment number which is this initial sequence number incre- 
mented by one. At this point connection 100 has been 
established between the browser and the load balancer. 

25 The toad balancer saves all of the SYN and ACK packet 
information received. Since incoming packets are few and 
small in size, a large amount of storage is not necessary. The 
SYN and ACK packets are later played back to the assigned 
server for TCP state migration. 

30 The browser then sends the first URL request 102 in a 
PUSH packet. A PUSH packet is identified by a PUSH flag 
being set in the TCP header. The load balancer parses the 
URL to get the file or resource name. Based on the requested 
resource, and the location of each resource in the web site, 

35 the load balancer determines which servers can serve the 
request, and then chooses the least busy of these servers as 
the assigned server. 

The load balancer then transfers the condition or state of 
the connection to the assigned server in a process the 

40 inventors call "TCP state migration". TCP state migration is 
not visible to the browser since all transactions occur 
between the load balancer and the assigned server. The 
assigned server also uses the virtual IP address so the 
browser is not aware that the SYN/ACK came from the load 

45 balancer while the data comes from the assigned server. 
TCP state migration 120 is performed by the load balancer 
playing back the SYN packet received from the browser and 
stored by the load balancer. The server responds with a 
SYN/ACK packet, which is intercepted and directed to the 

50 load balancer and not sent to the browser. The load balancer 
then sends the browser's stored ACK packet to the assigned 
server, and the assigned server is then connected directly to 
the browser, having the same TCP state as was established 
with the load balancer. 

55 The load balancer then sends the packet(s) containing the 
URL request to the assigned server as packet PUSH(O)'. The 
server reads its local disk and sends the requested file to the 
browser as data transfer 104. 

The load balancer then enters a pass- through state. Any 

60 further packets from the browser such as ACK packets for 
the received packets are passed through to the assigned 
server. The assigned server normally closes the connection 
immediately after the data has been sent to the browser. 
When the browser uses a "keep alive" mode, the server 

65 may not immediately close the connection. Additional 
requests may be sent to the assigned server. FIG. 11B shows 
the browser sending a second URL request, PUSH(l), to the 
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load balancer, which is passed through to the assigned server 
as PUSH(l)'. The second request is then handled by the 
server by sending the requested files to the browser, step 
126. Further requests are handled in a similar manner. 

Should one of these subsequent requests be for a file not 5 
located on the assigned server's local disk, a local network 
can be used by the assigned server to access the requested 
files from other servers on the local network. 

Finally, the server closes the connection 130 by sending 
a finish (FIN) packet which is intercepted and sent to the 10 
load balancer, which passes it through to the client, The 
client replies with an acknowledgment, ACK, and a FIN 
packet indicating that the browser is closing its end of the 
connection. The load balancer removes the session entry and 
the server assignment, and the assigned server closes the is 
connection. 

Protocol and IP Address Changes— FIGS. 12, 13 

FIG. 12 is a diagram of network layers showing a packet 
sent from the client to the server which is intercepted by the 
load balancer. A browser running on the application layer 75 20 
of the client sends packets to a web site to establish a 
connection and to send the URL requesting a file. The 
browser application sends the data to be sent, the URL 
addressed to the virtual IP address V of the web site, to 
TCP/IP stack 72. TCP/IP stack 72 is a stack of the standard 25 
TCP and IP layers, which are roughly equivalent to the 
session, transport, and network layers of the ISO network 
definition. TCP/IP stack packetizes the URL and adds head- 
ers and sends packets to link layer 74, which is the driver for 
the network-interface controller (NIC). Link layer 74 sends 30 
the packets out to Internet 66, perhaps through several 
routers, hubs, or switches (not shown). 

1he TCP/IP packets sent from the client have the virtual 
IP address V of the entire web site as their destination. These 
packets are shown as "TCP(V)" on the diagram. 35 

The TCP(V) packets are received by the web site and 
routed to the node having the load balancer since the packets 
have the virtual IP address. These packets are received by 
the low-level link layer 84 and sent up to modified TCP/IP 
stack 82. Modified TCP/IP stack 82 contains the standard 40 
TCP and IP modules with some modifications explained 
later. One modification is that incoming packets from the 
Internet have their protocol changed from TCP to a propri- 
etary "IXP" protocol. Since this IXP protocol is unknown to 
the standard TCP and IP layers, it is sent directly up to 45 
application layer 80 containing the load balancer. 

The load balancer in application layer 80 receives the 
modified IXP(V) packets with the IXP protocol and the 
virtual IP address V, and records the packets until the URL 
is received when it performs load balancing. The packets are 50 
then played back to the assigned server to perform TCP state 
migration as discussed in FIG. 11A. The packets played back 
are sent from application layer 80 as IXP(S) packets, having 
the IXP protocol and the assigned server's IP address S. 
Modified TCP/IP stack 82 uses the server's IP address S to 55 
determine the NIC address and the routing information from 
network routing tables. Once the routing has been deter- 
mined and the NIC address of the assigned server is found, 
modified TCP/IP stack 82 changes the protocol back to TCP 
and the IP address back to the virtual IP address V. The 60 
packets sent from modified TCP/IP stack 82 to link layer 84 
are TCP(V) packets. Link layer 84 sends these packets out 
over media 76 with the NIC address that corresponds to the 
assigned server, but with the virtual IP address as its desti- 
nation. 65 

The NIC address routes the TCP(V) packets to the 
assigned servers link layer 94, which passes the packets up 



to TCP/IP layer 92, perhaps after comparing checksums for 
error detection. TCP/IP layer 92 is also modified to route 
SYN, ACK, RST, and FIN packets back to scheduler appli- 
cation layer 80 so the status of the connection can be 
monitored, as explained later. Normal data transfers to 
browsers use PUSH packets which are not routed to sched- 
uler application layer 80. 

TCP/IP layer 92 recognizes the virtual IP address as a 
local IP address, so the packets are transferred up to appli- 
cation layer 90 after passing through the IP and TCP layers. 
Application layer 90 runs the httpd server software which is 
the standard web-site server software. Thus modified appli- 
cation server software is not needed. 

FIG. 13 is a diagram of network layers transmitting data 
packets from the server to the client browser. The server 
application in application layer 90 reads the files requested 
by the URL packets received in FIG. 12, and sends this data 
to TCP/IP layers 92 addressed to the IP address of the client, 
which was the source IP address of the incoming packets and 
was not changed. These packets use the TCP protocol and 
the client's IP address C, and are designated "TCP(C)". 
TCP/IP layer 92 uses the virtual IP address as the source 
address of these outgoing packets rather than the real IP 
address of the server. Since the incoming packets have the 
virtual IP address V as the destination IP address, the 
standard server software simply uses the destination IP 
address of incoming packets as the source IP address of 
outgoing packets. Link layer 94 receives these packets and 
sends them out over the local media 76 and eventually 
through Internet 66 to the client's link layer 74. These 
packets are sent up through the client's TCP/IP stack 72 and 
to the browser in application layer 75 where they are 
displayed to the user. 

Since the client received the virtual IP address as the 
source IP address in these packets, the client is unaware that 
the server is a different machine than the load-balancer, or 
other servers at the web site. 
Modified IP layer— FIG. 14 

FIG. 14 is a diagram of modifications to the IP layer for 
the load balancer's node. 

Unmodified link layer 84 passes packets received up to 
TCP/IP stack 82, and specifically to IP input module 200 of 
the IP layer. IP input module 200 determines if the packet is 
destined for the local node or must be routed or forwarded 
to another node. Forwarded packets are transferred to IP 
forward module 202, which prepares the packet for forward- 
ing. Routing tables are consulted by forwarding module 202 
to determine where to sent the packet next. Forwarded 
packets are then sent to IP output module 206, which sends 
them down to link layer 84. 

Local packets are assembled together for IP datagrams, 
which are passed up to TCP module 218 when the packet's 
type or protocol is TCP. TCP module 218 transmits the 
datagram to applications 212 in application layer 80 using 
TCP socket 216. 

Local packets that are not of a known protocol such as 
TCP or UDP (User Datagram Protocol) have an unrecog- 
nized protocol. These datagrams are sent to raw socket 214, 
bypassing TCP module 218. Any applications in application 
layer 80 can listen to raw socket 214 and use the datagram, 
since raw sockets are a standard TCP/IP feature. Load 
balancer 70 is an application which listens to raw socket 214 
for datagrams using the "IXP' protocol. Since the IXP 
protocol is not a defined protocol, no other applications 
should be looking for IXP datagrams. Thus using the IXP 
protocol allows use of raw socket 214 to bypass the TCP 
layer and send the datagrams directly to load balancer 70. 
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These datagrams are the connection packets and the URL Ordinary TCP packets which are received from IP forward 

originally from the client's browser. module 202 or from an application and passed down through 

Each server is modified to accept packets using the virtual the TCP layer have their destination IP address read, step 

IP address by aliasing a second IP address, thus using two IP 330, and their route determined by accessing routing tables, 

addresses. For example, in UNIX, the command: s step 332. The NIC address of the destination or the next hop 

% ifconfig deO 230.101.17.200 alias netmask Oxffffffff to the destination is determined and prefixed to the packet, 

specifies that a second IP address, the virtual IP address Ordinary TCP packets fail step 334 and are sent to the link 

230.101.17.200 is also an IP address for the node. Other layer, step 344. The link layer then transmits them over the 

operating systems also support IP address aliasing. media, unless the NIC address is the local node. These 

Modified IP Input Module — FIG. 15 10 local-destination packets are sent back up through the TCP/ 

FIG. 15 is a flowchart for a modified IP layer input IP stack, perhaps to a different application, 

module. The server with the load balancer uses modified IP Packets which have come from the load balancer appli- 

input module 200. An asterisk is used to indicate that the cation have the unrecognized IXP protocol rather than TCP, 

module is modified from the generic ip_input() module. and are diverted by step 334 for special processing. When 

Steps 308, 310, 312, and 314 are added steps which are not 15 the load balancer assigns the server at the local node, step 

in the generic IP module. 336 detects that the destination is the local node, and the 

All packets received from the media by the lower link packets are passed to step 340, with the IXP protocol intact, 

layer are passed up to the IP layer which calls IP input These IXP packets are passed back up to modified IP input 

module 200. Step 302 tests to determine if the packet is for module 200 and are detected as being for the local server as 

the local node by reading the destination IP address. 20 discussed in the next section. 

When step 302 determines that the destination IP address Packets assigned to another node's server must be sent 

is not a local IP address, then the packet is being routed over the media to the assigned server. Since the assigned 

through the local node and the IP layer acts as a software servers all use standard TCP/IP stacks, the IXP protocol 

router. The packet is passed to IP forward module 202 (step must be changed back to TCP, step 338. The destination IP 

304) which prepares the packet for forwarding. The packet 2s address is set to the destination server's real IP address by 

is then sent to IP output module 206 before being the load balancer so that the destination's NIC address is 

re-transmitted out the fink layer to the destination or the next generated by step 332 and prefixed to the packet. Since the 

hop. NIC address has already been determined, the IP address can 

Step 302 determines that the packet is for the local node safely be changed back to the virtual IP address of the entire 

when the IP address is the virtual IP address or the real IP 30 web site, step 340. The destination servers are all configured 

address for the server. The packet is stripped of its header to accept packets for the virtual IP address as well as for their 

information and possibly assembled with other packets to real IP address. Thus the real NIC address routes these 

form the IP datagram, step 306. packets to their destinations, which accepts them. 

The assembled IP datagram from step 306 is normally Modified Network Software for Wide -Area Network Sup- 
sent up to the TCP layer (steps 316, 318) for the generic IP 35 port 

module. The invention performs additional steps before step When the assigned server resides on the far side of a 
306 by modifying the generic IP input module to form router or across a wide -area network (WAN), the previous 
modified IP input module 200. Modified IP input module embodiment does not allow packets to get to their final 
200 checks the protocol to determine if it is the IXP protocol. destination. The load balancer normally transmits packets 
Since incoming packets from the Internet always use the 40 with the physical NIC address of the assigned server, but 
TCP protocol, incoming packets fail step 308 and are then with the virtual IP address. When the next hop is not the 
tested by step 310 to determine if they are TCP packets with assigned server, such as when the load balancer and the 
the virtual IP address and are world-wide-web packets. Thus assigned server are separated by a router, the router would 
step 310 looks for incoming packets. These incoming pack- route the packet back to the load balancer since the packets 
ets have their protocols changed from TCP to IXP, step 314. 45 have the load balancer's virtual IP address. 
The IXP protocol is not a recognized protocol, so step 316 FIG. 17 highlights that the real IP address of the assigned 
causes these incoming packets to be sent to the raw socket, server is used when multiple hops are required. Multiple- 
step 320, so that the load balancer application can read these hop data transmissions to the assigned server are supported 
packets. Thus changing the protocol to the unrecognized by further modifying the network software. The destination 
IXP protocol forces the incoming packets to be sent directly 50 IP address of the packets from the load balancer to the 
to the load balancer. This allows all incoming packets from assigned server are further modified to have the assigned 
the Internet to be routed through the load balancer. server's real IP address S rather than the virtual IP address 

Other TCP packets which are not world-wide web packets V. Thus intermediate routers can use the real IP address S of 

fail step 310 and are not modified. These ordinary TCP the assigned server to route the packet to the assigned server, 

packets are a known protocol, step 316, and are sent to the 55 When packets require multiple hops to reach the assigned 

TCP layer, step 318. server, the physical address of the next hop, intermediate 

Step 308, which first checks for the IXP protocol, is used router 97, is determined as before, but the real IP address of 

when the local node contains both the load balancer and the the assigned server is retained in the destination IP address 

assigned server. This step can be deleted if the local node is field of the IP header. To communicate the virtual address to 

used exclusively for the load balancer and cannot be an 60 the assigned server, the virtual IP address is appended to the 

assigned web server. end of the data in the packet and is sent to the assigned server 

Modified IP Output Module — FIG. 16 using the IXP protocol. Packets with the IXP protocol are 

FIG. 16 is a flowchart of an IP layer output module which intercepted and recovered by the assigned server, 

is modified for transmitting packets from the load balancer. A comparison of FIG. 17 to FIG. 12 shows that transmis- 

Modified IP output module 206 is a standard IP output 65 sion from load balancer's modified TCP/IP stack 82 to the 

module except that steps 334, 336, 338, and 340 have been assigned server's TCP/IP layer 92 uses IXP(S) packets 

added. rather than TCP(V) packets. Packets transmitted from the 
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load balancer to the assigned server are transmitted from 
link layer 84 of the load balancer over physical media 76 to 
intermediate router 97, then over physical media 76' to link 
layer 94 of the assigned server. Thus the local network at the 
server farm can use intermediate routers between the load 
balancer and the assigned servers. The assigned servers can 
also be located remotely from the load balancer, such as over 
a WAN using this technique. 
Added Steps When Assigned Server is Local 

Incoming packets which are assigned to the load balancer 
node's server are passed up and down the local TCP/IP stack 
twice. These packets are first sent from the low-level link 
layer through the modified IP layer to the load balancer in 
the application layer, and then back down through the IP 
layer to the link layer. Step 336 of FIG. 16 detects that the 
local server is the destination and bypasses steps 338, 340 so 
that the protocol is left as IXP. 

The link layer recognizes that the NIC address is the local 
NIC address and does not transmit the packets. Instead the 
packets are sent back up to the IP layer. Step 308 of FIG. 15 
detects these packets and changes the protocol back to TCP 
(step 312) and then passes the TCP packets to the HTTPD 
server application through the generic TCP layer. This 
sequence only occurs for a packet that has been intercepted 
to the load balancer and assigned to the server on the local 
node. 

IP Layer Modified for Servers 

The IP layers of the servers are modified in a similar way 
as the IP layer of the load balancer. All incoming packets are 
not affected, only special outgoing packets. These special 
packets are for establishing or closing a connection. These 
packets need to be intercepted and received by the load 
balancer during TCP state migration and when the session is 
closed. The load balancer keeps track of the session with an 
entry in a session table, and this entry is removed when the 
session ends. 

When the server closes a connection, it sends out a packet 
with the FIN flag set in the TCP header. When a transmission 
error occurs, a reset packet may be sent with the RST flag 
set. Likewise, when the server responds to the load balancer 
with the SYN/ACK packet during TCP state migration, the 
SYN and ACK flags are set. Normal data transfers to the 
browser are PUSH packets and do not have the SYN, ACK, 
RST, or FIN flags set. 

The server's IP output module is modified to detect these 
special packets by looking at the SYN, ACK, RST, and FIN 
flags. If none of these TCP flags are set, normal IP process- 
ing occurs. However, if any of these special TCP flags are 
set, then modified IP processing occurs. The protocol is 
changed from TCP to a variation of IXP called IXCP, and the 
destination address (the browser's IP address) is stored and 
replaced with the load balancer's IP address. The IP layer 
looks up the load balancer's IP address in its routing tables 
and generates the NIC address for the load balancer's node. 
The browser's IP address which was stored then replaces the 
load balancer's IP address, and the packet, with the IXCP 
protocol, is sent to the load balancer. 

The load balancer's IP layer processes these IXCP packets 
as shown in FIG. 15, the tests of steps 308, 310 fail, so that 
these IXCP packets are sent up to the load balancer through 
the raw socket. The load balancer can then alter its session 
table. FIN and RST packets are changed to TCP and the 
destination IP address changed to the browser to send out. 
SYN/ACK are not re-transmitted to the client. 
Load Balancer — FIG. 18 

FIG. 18 is a flowchart of the operation of the load 
balancer. The load balancer is written for the application or 
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user layer so that it can be easily modified. Writing the load 
balancer for a lower layer such as the IP layer puts the 
complex load balancing functions in the kernel, and any 
code modification could require that the system be shut 
5 down and rebooted. Having the load balancer in the appli- 
cation layer simplifies node management and allows code 
modifications to occur with minimal interruption. For per- 
formance reasons the inventors anticipate putting the load 
balancer into the kernel after being sufficiently debugged. 

Incoming web packets from the Internet are passed up to 
load balancer 70 from modified IP input module 200 (FIG. 
15). The load balancer is sometimes called a scheduler since 
it assigns or schedules sessions from browser clients to a 
server. 

The connection is first established with the browser client, 
15 step 350, by exchanging packets as was explained in more 
detail in FIG. 11 A. These packets to the browser for estab- 
lishing the connection are TCP packets with the browser 
client's IP address; they are not affected by the modifications 
to the IP output module since the TCP protocol is designated 
20 for these connection packets. The incoming and outgoing 
connection packets are saved and later played back to the 
assigned server for TCP state migration, as was also shown 
in FIG. 11A. 

Once the connection is established with the client, load 

25 balancer 70 waits for the URL and then parses the URL to 
determine the requested resource. For more complex URL's, 
such as those containing coordinates of an icon, load bal- 
ancer 70 needs to decode the URL to determine what 
resource is being requested. This decoding is normally done 

30 by the HTTPD server software. The resource location is thus 
determined, step 352, and resource-based load balancing can 
now be performed, step 354. Load balancing occurs among 
those servers which contain the requested files or resource. 
The virtual IP address of the destination for each packet 

35 is changed to the real IP address of the assigned server, step 
356, before each packet played back to the assigned server 
is sent to modified IP output module 206. The real IP address 
is used to determine the real NIC address of the assigned 
server before the IP address is changed back to the virtual IP 

40 address by modified IP output module 206. AsendtoO call, 
step 358, is used to send the packets to modified IP output 
module 206. 

Fault-Tolerant Web Site— FIG. 19 

FIG. 19 is a diagram of a fault-tolerant web site with a 

45 back-up load balancer and dual Internet connections. 
Browser 10 sends requests through Internet 66 with a virtual 
IP address for the whole web site. Incoming packets with the 
virtual IP address are routed to load balancer 70 over local 
LAN 144. Local LAN 144 may contain routers, switches, 

50 and hubs when servers are located on separate network 
nodes. Local LAN 144 connects to Internet 66 through 
Internet connection 142 which directly connects to Internet 
connection router 140, and through Internet connection 148, 
which is connected to Internet connection router 146. 

55 Two separate connections 142, 148 are provided to Inter- 
net 66 to provide a backup when one connection fails, either 
due to fine problems or failure of a connection router or 
other hardware or software. Having two connections 
increases the bandwidth which is needed for larger web 

60 sites. Each connection 142, 148 may be a slower Tl tele- 
communications connection, which supports 1.5 mega bits 
per second (Mbps), or a more powerful T3 connection which 
supports 45.5 Mbps. Other Internet connections may be used 
and mixed freely. Additional Internet connections may be 

65 added as demand increases simply by adding another con- 
nection router to connect the new connection to local LAN 
144. 
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Since the load balancer is no longer in the router, several level software program in the application layer which is 

connection routers may be used and added or subtracted as more easily modified than Lower-level kernel modules, 

needed without affecting load balancing. Standard routers The standard TCP/IP layers are used except for modifi- 

and new technologies can be used. The connection router is cations to the IP layer. This is an advantage since standard 

not limited to having a certain operating system used by the 5 software is used as much as possible. The application layer 

load balancer. Should a connection router 140, 146 fail, a funs toe standard httpd server software. Modified server 

backup connection router can continue to connect the web software is not needed. Having the load balancer in the 

servers to Internet 66. There is no single point of failure. application layer simplifies node management and allows 

A backup load balancer 70* is also provided to take over code to occur ^ minimal interruption, 

operation should primary load balancer 70 fail. These load 10 ALTERNATE EMBODIMENTS 

balancers are located on separate servers to lessen the Several other embodiments are contemplated by the 

chance that both fail at the same time. Backup load balancer For example various local networks may be used, 

70' closely monitors primary load balancer 70 to detect a mc i u ding those which use token-ring or other NIC addresses 

failure- rather than MAC address as the local or physical network 

The content of the web site is not mirrored to each is address Switches, bubs, and hardware or software-based 

server's disk. However, to prevent the failure of one server routeIS can be inserted for larger net works. 

from making some content unavailable at least one backup £ach ^ process requests frQm muhiple cUentSj 

copy is made of each file m the web site. For example, especially when multitasking operating systems such as 

file.btml 26 existe on disk 62 attached to server 52 and a UN]X ^ windows m m ^ a connccti on 

backup copy of file hind 26* is located on disk 60 of server 20 h ^ ic hag been described? me rannec . 

51. Load balancer 70 maintains a table or other data state- ^ ^ ^ be madc ^ h rfvate Qetworks ^ as 

ture of all the locations of files m the web site which is used rate networks taam as Intranets . Intranets are just a 

for load balancing. Software utilities may be used to reph- sufeset of ^ . JntemV ^ the web site be 

cate new files or delete all copies of old files. Access behmd a rate fircwall and not be visiblc to the ^ 0 f 

statistics of each file or directory of files can be kept to 2s ^ i nlernet 

determine which files are more frequently used and should ^ ^ ^ be & „ web _ hoster „ containing many web 

be replicated to more servers, and which files are infre- fof ^ rather than a wcbsite . To 

quently used and only two copies should be kept. load balancef be 

Content A is located on disk 58 attached to server 56 PP ^ * 

which is used for bo h load balancing and as a server. 30 ^.^J, ^ Heterogeneous clusters could 

Content A is also located on disk 60 attached to 51. q{ smm ^ each ^ mnni 

Content B is located on disk 60 ►and disk 62 "J*™** Afferent operating Tystems (OS's) such as UNIX, Windows/ 

C is located on disk 58 and disk 63 accessed by server 55, ^ gQj^y^jg e * c 

which also contains backup load balancer 70'. ' ' * . (1 

A failure of one disk or one server does not crash the 35 ™ e ™*> / ann h« been described as having a local 

entire web site, or even make unavailable some of its b * ^ netw °* c °* ld b < loCal ^ m j h ° 

content, as aU intent is backed up on a. least one other f"» "»at it * not the Internet backbone. Servers » the web 

server. When a failure occurs, system maintenance software *™ a>ay be geographically remote where some of he 

makes another backup copy of the lost files so that a second se ™f are ° cat f d "J ° ne Clt y wbjle ° ther ^ a ' e ^ 

failure does not make some flies unavailable. 40 J» °^ cltie f - balan » n 8 be P erformed n °*^ 

Even a failure by load balancer 70 does not bring down base u d ° n °° nt ^ bu < ako Sfio&*P*™} X to minimize traffic 

the web site as backup load balancer 70' is ready to take over °° *>* ° etwork ba f bo ,? e ' ^ e Pf" 8 of the web T s '£ m a ?$> 

load balancing. A failure by the Internet connection router ™? be conne f * d ° ca »y tbrou S h ° ne or mo ^. 1 ^' 

also does nof lock out the entire web site as a backup be ' n S connected to other cities using a WAN. The IXP 

connection and router can be used. 45 P<° toco1 «» be /° r a11 P a <*ets sent from toe load 

balancer to the assigned server, even when multiple hops are 

ADVANTAGES OF THE INVENTION no t required. 

Request-level load-balancer granularity is provided by the The invention has been described as a browser accessing 
web site since a browser sees a failure only when the a file on the server, but the file may actually be a resource 
assigned server fails while processing the URL request. 50 which is not necessarily a traditional file. The file may be a 
Since the entire web site uses a virtual IP address, client program, applet, or script which is executed, or an interface 
caching of the IP address causes no problems, as all new into an SQL database or to fast or memory-intensive corn- 
requests are routed to the load balancer, or the backup load putational resource, Web servers support application- 
balancer when the primary load balancer fails. Programming Interfaces (API's) which enable servers to be 

The invention provides a highly fault-tolerant web site. 55 compiled with unique capabilities for alternate types of 

Such a web site greatly reduces the probability that a user server resources. 

gets the "SERVER NOT RESPONDING" error message Since these resources arc expensive to implement on all 

when a server fails at the web site. Request-level granularity servers, it is desired to allocate just one or a few servers to 

results in fewer browser users being caught when a server handling requests for these resources. The invention waits to 

crashes; only those users having a current request being eo perform load-balancing and assign a server until after the 

served by that server experience an error. Other requests in URL has been parsed. Thus the invention is ideal for 

a session at that web site, and future sessions are not assigning specialty servers having these resources. The web 

affected. site can be segregated by resource and still be load-balanced. 

Servers and routers and even Internet connections can be The invention may also be applied to other Internet traffic 

added or removed without bringing down the web site. Thus 65 as well. The in vention could be applied to a File-Transfer- 

a highly maintainable and expandable web site is continu- Protocol (FTP) server, a rlogin server, or a telnet server, 

ously available to users. The load balancer itself is a user- either as a stand-alone site or as a part of a larger web site. 
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While delayed resource binding is preferred, other 
embodiments are contemplated. HTTP redirection from the 
scheduler to the assigned server may be used after the URL 
has been parsed. The load balancer sends the client the 
address of the assigned server and instructs the client to 
re -issue the URL request using the assigned server 1 s 
address. Thus redirection is delayed until the URL is parsed 
and the requested content is determined. 

The URL passed to web server can also be modified to 
reference relocated web pages without changing the links. 

The foregoing description of the embodiments of the 
invention has been presented for the purposes of illustration 
and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed. Many modifi- 
cations and variations are possible in light of the above 
teaching. It is intended that the scope of the invention be 
limited not by this detailed description, but rather by the 
claims appended hereto. 

We claim: 

1. A web site for sending resources to a browser on a client 
connected to a computer network, the web site comprising: 

a network connection point for receiving incoming data 
packets from the computer network and for transmit- 
ting outgoing data packets to the computer network; 

local network, coupled to the network connection point, 
for transferring data packets; 

a plurality of network nodes containing web servers with 
resources, the plurality of network nodes connected to 
the local network, the plurality of network nodes 
including means for transmitting the resources as out- 
going data packets to the client, the plurality of network 
nodes including means for sending the outgoing data 
packets over the local network to the network connec- 
tion point; 

wherein the plurality of network nodes containing web 
servers together contain all resources at the web site, 
but each network node in the plurality of network nodes 
contains only a portion of all the resources at the web 
site; 

a balancer network node containing a load balancer, 
receiving the incoming data packets transmitted over 
the local network from the network connection point, 
the load balancer for determining an assigned server in 
the plurality of network nodes for responding to a 
request from the client in an incoming data packet, the 
load balancer including means for transferring a con- 
nection to the client to the assigned server; 
wherein the balancer network node containing the load 
balancer is connected to the network connection point 
by the local network which is also connected to the 
plurality of network nodes, 
wherein network nodes are segregated to contain different 
resources, and wherein all resources at the web site are 
not mirrored to all network nodes at the web site, 
wherein the load balancer further comprises: 

content means for storing an indication of which net- 
work nodes in the plurality of network nodes contain 
each resource; 
URL means, receiving incoming data packets from the 
client containing a request for a resource, for deter- 
mining a requested resource from the incoming data 
packets; 

compare means, coupled to the content means and 
coupled to the URL means, for comparing the 
requested resource to the indication of which net- 
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work nodes in the plurality of network nodes contain 
each resource, and for outputting a list of network 
nodes containing the requested resource; 
balancing means, receiving the list of network nodes 
containing the requested resource, for choosing as an 
assigned node one of the network nodes in the list of 
network nodes, 
whereby the incoming data packets are routed to the 
balancer network node but outgoing data packets 
bypass the balancer network node and whereby the load 
balancer chooses an assigned node based on the 
resources contained by each network node, the load 
balancer performing resource-based load balancing, 

2. The web site of claim 1 wherein the balancer network 
node is in the plurality of network nodes containing web 
servers. 

3. The web site of claim 1 wherein the web site is 
addressable by one network address for all web servers in 
the plurality of network nodes containing web servers. 

4. The web site of claim 1 further comprising: 

delay means, in the load balancer, for delaying assignment 
of the assigned node until an incoming data packet 
containing the request for the resource is received, 

whereby load balancing is delayed. 

5. The web site of claim 1 further comprising: 
redirect means, in the load balancer, for directing the 

client to issue a new URL request directly to the 
assigned node using an address of the assigned node 
provided by the load balancer to the client; 
whereby the client is redirected to the assigned server by 
the load balancer. 

6. A computer-implemented method of servicing requests 
for resources from a client by nodes containing different 
resources, the computer-implemented method comprising 
the steps of: 

making a connection and setting up a session between the 
client and a load balancer at a web site for servicing 
requests from clients; 

waiting for a URL request from the client once the load 
balancer has made the connection with the client; 

receiving the URL request from the client and decoding 
the URL request to determine a requested resource; 

comparing an identifier for the requested resource to 
identifiers for resources located on a plurality of nodes 
and determining a first subset of the plurality of nodes 
which contain the requested resource and a second 
subset of the plurality of nodes which do not contain the 
requested resource; 

assigning the URL request to an assigned node in the first 
subset of the nodes which contain the requested 
resource, by determining the assigned node to be a 
server in the first subset of the nodes which is least busy 
processing requests, wherein the assigned node is not in 
the second subset; 

transferring the connection and the session setup to the 
. assigned node containing the requested resource by 
storing packets received from the client when estab- 
lishing the connection and by transmitting the pack- 
ets to the assigned node after the URL request is 
received; 

reading the requested resource on the assigned node and 
transmitting the requested resource to the client, 

whereby the assigned node is selected based on a location 
of the requested resource determined from the URL 
request and load balancing is performed among nodes 
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having the requested resource and the connection is 
transferred from the load balancer to the assigned node 
by re-transmitting the packets to the assigned node. 

7. The computer-implemented method of claim 6 wherein 
the packets received from the client are TCP/IP packets 5 
having a destination IP address being a virtual IP address of 
the load balancer, and wherein the step of transmitting the 
packets to the assigned node comprises: 

changing the virtual IP address of the load balancer in the 
packets to a real IP address of the assigned node and 10 
passing the packets to a modified IP layer; 

determining from the real IP address a physical route from 
the load balancer to the assigned node over a network 
and generating a physical network address for the 
assigned node and attaching the physical network 15 
address to the packets; 

changing the real IP address in the packets back to the 
virtual IP address before transmission of the packets 
with the physical network address, 2Q 

whereby the physical network address is generated from 
the real IP address of the assigned node, but the packets 
transmitted to the assigned node contain the virtual IP 
address of the load balancer. 

8. The computer-implemented method of claim 6 wherein ^ 
the packets received from the client are TCP/IP packets 
having a destination IP address being a virtual IP address of 
the load balancer, and wherein the step of transmitting the 
packets to the assigned node comprises: 

changing the virtual IP address of the load balancer in the 30 
packets to a real IP address of the assigned node and 
passing the packets to a modified IP layer; 

determining from the real IP address a physical route from 
the load balancer to an intermediate router in a path to 
the assigned node over a network and generating a 35 
physical network address of the intermediate router and 
attaching the physical network address of the interme- 
diate router to the packets; and 

transmitting packets having the real IP address of the 
assigned node as the destination IP address and the 40 
virtual IP address of the load balancer appended to data 
in the packet; 

recovering the virtual IP address of the load balancer from 
the data in the packet when the packet is received by the 
assigned node, 

whereby the physical network address of the intermediate 
router is generated from the real IP address of the 
assigned node, the load balancer and the assigned node 
being separated by the intermediate router. 

9. The computer-implemented method of claim 7 wherein 50 
the load balancer is a program in an application layer above 

a TCP layer which is above the modified IP layer which is 
above a link layer, wherein the step of receiving the URL 
request from the client comprises: 

receiving at least one TCP/IP packet from the client and 
assembling an IP datagram from the at least one TCP/IP 
packet in the modified IP layer; 
changing a protocol for the IP datagram from TCP to an 

unrecognized protocol; 60 
bypassing the TCP layer and transmitting the IP datagram 
to the load balancer in the application layer through a 
raw IP socket, 
whereby the TCP layer is bypassed for incoming TCP/IP 
packets of the URL request. 65 

10. The computer-implemented method of claim 9 
wherein the step of transferring the connection and the 
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session setup to the assigned node containing the requested 

resource further comprises: 

passing the packets with the virtual IP address up through 
a modified IP layer and a standard TCP layer to a 
standard server application in an application layer on 
the assigned node, the assigned node being configured 
to accept packets with either the real IP address of the 
assigned node or the virtual IP address of the load 
balancer, 

whereby the assigned node uses the modified IP layer and 
the standard server application. 

11. The computer-implemented method of claim 10 
wherein the step of transmitting the requested resource to the 
client from the assigned node comprises 

transmitting the requested resource in TCP/IP outgoing 
packets which contain the virtual IP address of the load 
balancer as a source IP address but an IP address for the 
client as the destination IP address, wherein the TCP/IP 
outgoing packets bypass a node with the load balancer, 

whereby incoming packets are routed to the load balancer 
but the outgoing packets bypass the node with the load 
balancer. 

12. The computer-implemented method of claim 11 fur- 
ther comprising the steps of: 

creating a session entry for the client in the load balancer 
when the URL request from the client is received by the 
load balancer; 

updating the session entry for the client to indicate the 
assigned node when the load balancer assigns the URL 
request to the assigned node, 

whereby the load balancer tracks sessions between clients 
and assigned nodes. 

13. The computer-implemented method of claim 12 fur- 
ther comprising the steps of: 

reading a FIN flag in the TCP/IP outgoing packets and 

determining that the TCP/IP outgoing packet is a FIN 

packet when the FIN flag is set; 
changing the IP address of the client to the virtual IP 

address of the load balancer as the destination IP 

address for the FIN packet; 
transmitting the FIN packet to the load balancer and 

closing the session entry for the client in the load 

balancer in response to the FIN packet; 
re-transmitting from the load balancer the FIN packet to 

the client, 

whereby FIN packets are intercepted by the load balancer. 

14. A fault-tolerant server farm for serving resources to 
browser clients remotely located on a network, the resources 
containing links to other resources not located at the server 
farm but located on distant computers on the world-wide 
web, each link being a universal-resource locator (URL), the 
URL indicating a host name and a requested resource, the 
host name indicating a server farm on the network contain- 
ing the requested resource, the fault-tolerant server farm 
comprising: 

a network connection for transferring packets from the 
network to a local network; 

a plurality of nodes, each node being a computer con- 
taining a disk and a connection to the local network; 

a plurality of frequently-accessed resources stored on the 
disk for each node; 

a plurality of less-frequently-accessed resources, each of 
the less-frequently -accessed resources stored on disks 
for at least two nodes but not stored on the disk for each 
node; 
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a primary Load balancer, residing on a primary node in 
the plurality of nodes, for receiving all incoming 
packets from the network connection, the primary 
load -balancer assigning URL requests from browser 
clients to nodes in the plurality of nodes, wherein the 
primary load balancer comprises: 
storage means for storing at least a portion of con- 
nection incoming packets for establishing a con- 
nection between a browser client and the server 
farm; 

reply means for generating acknowledgment packets 
to the browser client in response to the connection 
incoming packets; 

URL decoder means, receiving a URL packet once 
the connection with the browser client is made, for 
decoding the URL to determine a requested 
resource requested by the browser client; 

assignment means for selecting an assigned node in 
the plurality of nodes by not selecting nodes which 
have disks which do not contain the requested 
resource; 

transfer means for transferring the connection to the 
assigned node by constructing packets using the 
storage means which stored at least a portion of 
connection incoming packets; 

pass-through means for transferring incoming pack- 
ets from the browser client to the assigned node 
once the connection has been transferred to the 
assigned node, 
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a secondary load balancer, residing on a secondary node 
in the plurality of nodes, for receiving all incoming 
packets from the network connection when the primary 
load balancer fails, the secondary load-balancer assign- 
ing URL requests from browser clients to nodes in the 
plurality of nodes, 

whereby each node does not contain all resources at the 
server farm and the primary and secondary load bal- 
ancers reside on nodes connected to the local network. 

15. The fault-tolerant server farm of claim 14 further 
comprising: 

balancing means, coupled to the primary load balancer 
and to the secondary load balancer, for assigning con- 
nection incoming packets to either the primary load 
balancer or to the secondary load balancer, 

whereby load balancing is distributed between the pri- 
mary load balancer and the secondary load balancer. 

16. The fault- tolerant server farm of claim 15 wherein the 
network is the Internet, the fault-tolerant server farm further 
comprising: 

a secondary Internet connection for transferring packets 
from the Internet to a local network, 

whereby two Internet connections connect the local net- 
work to the Internet. 
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